hallucination in LLMs

Large-language models may sometimes hallucinate, that is generate apparatently believable facts, or references to literature that are entirely fictitious. This is not borne out of any explicit intention to deceive, merely that the LLM is trained on large volumes textual material, and so can produce realistic text without any substance behind them. Techniques such as RAG (retrieval-augmented generation}}, where sources are identified as part of or alongside text generation, can reduce the rosk of hallucination. Alternatively output text can be processed and verified by more knowledge-rich reasoning.

In some ways LLM hallucinations are similar to confabulation in humans, where gaps in memory or filled in by the the brain by plausible, but not necessarily accurate details.